{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f6118914",
   "metadata": {},
   "source": [
    "# SqueezeNet 总结\n",
    "\n",
    "[SqueezeNet](https://arxiv.org/abs/1602.07360)  是由 DeepScale、UC Berkeley 和 Stanford 联合提出的一种轻量级卷积神经网络，旨在在保持与 AlexNet 相当精度的同时，大幅减少模型参数数量。它非常适合应用于资源受限的设备（如移动端或嵌入式系统）。\n",
    "\n",
    "## 关键创新点\n",
    "\n",
    "### 1. **Fire Module**\n",
    "SqueezeNet 的核心结构是 **Fire Module**，由两个部分组成：\n",
    "\n",
    "- **Squeeze Layer**：使用 1x1 卷积核进行降维，减少输入通道数。\n",
    "- **Expand Layer**：并行使用 1x1 和 3x3 卷积核提取特征。\n",
    "\n",
    "```python\n",
    "class Fire(nn.Module):\n",
    "    def __init__(self, inplanes: int, squeeze_planes: int, expand1x1_planes: int, expand3x3_planes: int) -> None:\n",
    "        super().__init__()\n",
    "        self.inplanes = inplanes\n",
    "        self.squeeze = nn.Conv2d(inplanes, squeeze_planes, kernel_size=1)\n",
    "        self.squeeze_activation = nn.ReLU(inplace=True)\n",
    "        self.expand1x1 = nn.Conv2d(squeeze_planes, expand1x1_planes, kernel_size=1)\n",
    "        self.expand1x1_activation = nn.ReLU(inplace=True)\n",
    "        self.expand3x3 = nn.Conv2d(squeeze_planes, expand3x3_planes, kernel_size=3, padding=1)\n",
    "        self.expand3x3_activation = nn.ReLU(inplace=True)\n",
    "\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        x = self.squeeze_activation(self.squeeze(x))\n",
    "        return torch.cat(\n",
    "            [self.expand1x1_activation(self.expand1x1(x)), self.expand3x3_activation(self.expand3x3(x))], 1\n",
    "        )\n",
    "```\n",
    "\n",
    "这种模块设计有效减少了参数数量，同时保留了丰富的特征表达能力。\n",
    "\n",
    "### 2. **大量使用 1x1 卷积**\n",
    "相比传统的 3x3 或更大卷积核，1x1 卷积可以显著减少计算量和参数量，同时实现跨通道的信息整合。\n",
    "\n",
    "### 3. **延迟下采样（Delayed Downsampling）**\n",
    "通过在网络前几层避免过早地进行下采样（如 Strided Convolution），提升了最终特征图的空间分辨率，有助于提高准确率。\n",
    "\n",
    "### 4. **模型小型化**\n",
    "SqueezeNet 最终模型大小可压缩到小于 **0.5MB**，仅为 AlexNet 的 1/50 左右，非常适合部署在存储受限的设备上。\n",
    "\n",
    "### 5. **易于压缩**\n",
    "SqueezeNet 在提出时就考虑了后续模型压缩的可能性（如使用 Deep Compression 技术），可以在几乎不损失精度的前提下进一步减小模型体积。\n",
    "\n",
    "![alt text](resources/squeezenet_arch.png \"Title\")\n",
    "\n",
    "---\n",
    "\n",
    "## 缺点与局限性\n",
    "\n",
    "### 1. **推理速度未必更快**\n",
    "虽然参数少，但由于频繁使用多个卷积操作（尤其是在 Fire Module 中的并行分支），实际推理速度不一定比结构更简单的模型快。\n",
    "\n",
    "### 2. **对硬件优化要求高**\n",
    "为了充分发挥其轻量优势，需要良好的硬件支持和高效的卷积实现（例如针对 1x1 卷积的优化）。\n",
    "\n",
    "### 3. **精度上限有限**\n",
    "尽管在轻量模型中表现不错，但在 ImageNet 等大型数据集上的 Top-1 准确率仍落后于更深、更复杂的模型（如 ResNet、DenseNet 等）。\n",
    "\n",
    "### 4. **扩展性不如现代轻量化模型**\n",
    "相比 MobileNet、ShuffleNet 等后来提出的轻量级模型，SqueezeNet 在移动端优化和性能平衡方面略显不足。\n",
    "\n",
    "---\n",
    "\n",
    "## 总结\n",
    "\n",
    "SqueezeNet 是早期探索轻量级 CNN 的代表作之一，其 Fire Module 设计理念为后续模型提供了重要启发。尽管存在一些性能瓶颈，但其在模型压缩和部署方面的潜力仍然值得肯定，尤其适用于对模型体积敏感的应用场景。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4b91c8b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Use device:  cuda\n"
     ]
    }
   ],
   "source": [
    "# 自动重新加载外部module，使得修改代码之后无需重新import\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "from hdd.device.utils import get_device\n",
    "from hdd.dataset.imagenette_in_memory import ImagenetteInMemory\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torchvision import datasets, transforms\n",
    "\n",
    "# 设置训练数据的路径\n",
    "DATA_ROOT = \"~/workspace/hands-dirty-on-dl/dataset\"\n",
    "# 设置TensorBoard的路径\n",
    "TENSORBOARD_ROOT = \"~/workspace/hands-dirty-on-dl/dataset\"\n",
    "# 设置预训练模型参数路径\n",
    "TORCH_HUB_PATH = \"~/workspace/hands-dirty-on-dl/pretrained_models\"\n",
    "torch.hub.set_dir(TORCH_HUB_PATH)\n",
    "# 挑选最合适的训练设备\n",
    "DEVICE = get_device([\"cuda\", \"cpu\"])\n",
    "print(\"Use device: \", DEVICE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e26d99aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "from hdd.data_util.transforms import RandomResize\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "TRAIN_MEAN = [0.4625, 0.4580, 0.4295]\n",
    "TRAIN_STD = [0.2452, 0.2390, 0.2469]\n",
    "train_dataset_transforms = transforms.Compose(\n",
    "    [\n",
    "        RandomResize([256, 296, 384]),  # 随机在三个size中选择一个进行resize\n",
    "        transforms.RandomRotation(10),\n",
    "        transforms.RandomCrop(224),\n",
    "        transforms.RandomHorizontalFlip(),\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize(mean=TRAIN_MEAN, std=TRAIN_STD),\n",
    "    ]\n",
    ")\n",
    "val_dataset_transforms = transforms.Compose(\n",
    "    [\n",
    "        transforms.Resize(256),\n",
    "        transforms.CenterCrop(224),\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize(mean=TRAIN_MEAN, std=TRAIN_STD),\n",
    "    ]\n",
    ")\n",
    "train_dataset = ImagenetteInMemory(\n",
    "    root=DATA_ROOT,\n",
    "    split=\"train\",\n",
    "    size=\"full\",\n",
    "    download=True,\n",
    "    transform=train_dataset_transforms,\n",
    ")\n",
    "val_dataset = ImagenetteInMemory(\n",
    "    root=DATA_ROOT,\n",
    "    split=\"val\",\n",
    "    size=\"full\",\n",
    "    download=True,\n",
    "    transform=val_dataset_transforms,\n",
    ")\n",
    "\n",
    "\n",
    "def build_dataloader(batch_size, train_dataset, val_dataset):\n",
    "    train_dataloader = DataLoader(\n",
    "        train_dataset, batch_size=batch_size, shuffle=True, num_workers=8\n",
    "    )\n",
    "    val_dataloader = DataLoader(\n",
    "        val_dataset, batch_size=batch_size, shuffle=False, num_workers=8\n",
    "    )\n",
    "    return train_dataloader, val_dataloader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7db16edd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#Parameter: 727626\n",
      "Epoch: 1/250 Train Loss: 6.4859 Accuracy: 0.1574 Time: 6.43627  | Val Loss: 2.1918 Accuracy: 0.1740\n",
      "Epoch: 2/250 Train Loss: 2.1915 Accuracy: 0.1795 Time: 6.24756  | Val Loss: 2.1425 Accuracy: 0.2003\n",
      "Epoch: 3/250 Train Loss: 2.1748 Accuracy: 0.1951 Time: 6.22408  | Val Loss: 2.1112 Accuracy: 0.2191\n",
      "Epoch: 4/250 Train Loss: 2.1465 Accuracy: 0.2179 Time: 6.21362  | Val Loss: 2.0906 Accuracy: 0.2306\n",
      "Epoch: 5/250 Train Loss: 2.0998 Accuracy: 0.2355 Time: 6.28024  | Val Loss: 2.0212 Accuracy: 0.2810\n",
      "Epoch: 6/250 Train Loss: 2.0417 Accuracy: 0.2709 Time: 6.22656  | Val Loss: 1.9415 Accuracy: 0.2971\n",
      "Epoch: 7/250 Train Loss: 2.0223 Accuracy: 0.2744 Time: 6.32947  | Val Loss: 1.9426 Accuracy: 0.3057\n",
      "Epoch: 8/250 Train Loss: 2.0173 Accuracy: 0.2756 Time: 6.23133  | Val Loss: 1.9583 Accuracy: 0.3126\n",
      "Epoch: 9/250 Train Loss: 2.0000 Accuracy: 0.3018 Time: 6.23780  | Val Loss: 1.9481 Accuracy: 0.2973\n",
      "Epoch: 10/250 Train Loss: 1.9419 Accuracy: 0.3363 Time: 6.29902  | Val Loss: 1.7987 Accuracy: 0.4000\n",
      "Epoch: 11/250 Train Loss: 1.8844 Accuracy: 0.3602 Time: 6.34174  | Val Loss: 1.8686 Accuracy: 0.3521\n",
      "Epoch: 12/250 Train Loss: 1.8497 Accuracy: 0.3798 Time: 6.33018  | Val Loss: 1.7433 Accuracy: 0.4311\n",
      "Epoch: 13/250 Train Loss: 1.8029 Accuracy: 0.3958 Time: 6.34164  | Val Loss: 1.7380 Accuracy: 0.4224\n",
      "Epoch: 14/250 Train Loss: 1.7711 Accuracy: 0.4161 Time: 6.38840  | Val Loss: 1.6693 Accuracy: 0.4561\n",
      "Epoch: 15/250 Train Loss: 1.7110 Accuracy: 0.4348 Time: 6.31948  | Val Loss: 1.7004 Accuracy: 0.4367\n",
      "Epoch: 16/250 Train Loss: 1.6569 Accuracy: 0.4638 Time: 6.35775  | Val Loss: 1.6039 Accuracy: 0.4910\n",
      "Epoch: 17/250 Train Loss: 1.6725 Accuracy: 0.4572 Time: 6.31252  | Val Loss: 1.5562 Accuracy: 0.5164\n",
      "Epoch: 18/250 Train Loss: 1.5383 Accuracy: 0.4919 Time: 6.33523  | Val Loss: 1.3881 Accuracy: 0.5445\n",
      "Epoch: 19/250 Train Loss: 1.4036 Accuracy: 0.5456 Time: 6.40666  | Val Loss: 1.2168 Accuracy: 0.6046\n",
      "Epoch: 20/250 Train Loss: 1.3226 Accuracy: 0.5722 Time: 6.37492  | Val Loss: 1.3901 Accuracy: 0.5534\n",
      "Epoch: 21/250 Train Loss: 1.2733 Accuracy: 0.5894 Time: 6.34038  | Val Loss: 1.1298 Accuracy: 0.6390\n",
      "Epoch: 22/250 Train Loss: 1.2511 Accuracy: 0.6015 Time: 6.33959  | Val Loss: 1.1359 Accuracy: 0.6392\n",
      "Epoch: 23/250 Train Loss: 1.2622 Accuracy: 0.5916 Time: 6.35044  | Val Loss: 1.1221 Accuracy: 0.6482\n",
      "Epoch: 24/250 Train Loss: 1.1834 Accuracy: 0.6180 Time: 6.33108  | Val Loss: 1.0671 Accuracy: 0.6660\n",
      "Epoch: 25/250 Train Loss: 1.1773 Accuracy: 0.6192 Time: 6.36673  | Val Loss: 1.0384 Accuracy: 0.6724\n",
      "Epoch: 26/250 Train Loss: 1.1680 Accuracy: 0.6228 Time: 6.31761  | Val Loss: 1.0394 Accuracy: 0.6792\n",
      "Epoch: 27/250 Train Loss: 1.1365 Accuracy: 0.6335 Time: 6.44348  | Val Loss: 0.9839 Accuracy: 0.6948\n",
      "Epoch: 28/250 Train Loss: 1.1054 Accuracy: 0.6435 Time: 6.42404  | Val Loss: 1.0003 Accuracy: 0.6764\n",
      "Epoch: 29/250 Train Loss: 1.1151 Accuracy: 0.6432 Time: 6.38237  | Val Loss: 1.0311 Accuracy: 0.6680\n",
      "Epoch: 30/250 Train Loss: 1.1092 Accuracy: 0.6438 Time: 6.22832  | Val Loss: 0.9977 Accuracy: 0.6910\n",
      "Epoch: 31/250 Train Loss: 1.0923 Accuracy: 0.6486 Time: 6.30843  | Val Loss: 1.0090 Accuracy: 0.6818\n",
      "Epoch: 32/250 Train Loss: 1.0962 Accuracy: 0.6537 Time: 6.29135  | Val Loss: 0.9789 Accuracy: 0.6940\n",
      "Epoch: 33/250 Train Loss: 1.0399 Accuracy: 0.6645 Time: 6.32499  | Val Loss: 0.9571 Accuracy: 0.7009\n",
      "Epoch: 34/250 Train Loss: 1.0542 Accuracy: 0.6590 Time: 6.25238  | Val Loss: 0.9164 Accuracy: 0.7080\n",
      "Epoch: 35/250 Train Loss: 1.0283 Accuracy: 0.6765 Time: 6.34165  | Val Loss: 0.9270 Accuracy: 0.7017\n",
      "Epoch: 36/250 Train Loss: 1.0283 Accuracy: 0.6734 Time: 6.22465  | Val Loss: 0.9375 Accuracy: 0.7045\n",
      "Epoch: 37/250 Train Loss: 1.0153 Accuracy: 0.6771 Time: 6.29283  | Val Loss: 0.8821 Accuracy: 0.7169\n",
      "Epoch: 38/250 Train Loss: 1.0053 Accuracy: 0.6825 Time: 6.34651  | Val Loss: 0.9074 Accuracy: 0.7131\n",
      "Epoch: 39/250 Train Loss: 1.0013 Accuracy: 0.6802 Time: 6.14273  | Val Loss: 0.8877 Accuracy: 0.7195\n",
      "Epoch: 40/250 Train Loss: 0.9875 Accuracy: 0.6865 Time: 6.12626  | Val Loss: 0.9409 Accuracy: 0.6986\n",
      "Epoch: 41/250 Train Loss: 0.9965 Accuracy: 0.6804 Time: 6.17228  | Val Loss: 0.8705 Accuracy: 0.7213\n",
      "Epoch: 42/250 Train Loss: 0.9530 Accuracy: 0.6980 Time: 6.18971  | Val Loss: 0.8833 Accuracy: 0.7190\n",
      "Epoch: 43/250 Train Loss: 0.9423 Accuracy: 0.6997 Time: 6.16264  | Val Loss: 0.8398 Accuracy: 0.7310\n",
      "Epoch: 44/250 Train Loss: 0.9395 Accuracy: 0.7014 Time: 6.19585  | Val Loss: 0.9137 Accuracy: 0.7121\n",
      "Epoch: 45/250 Train Loss: 0.9467 Accuracy: 0.6994 Time: 6.15436  | Val Loss: 0.9163 Accuracy: 0.7080\n",
      "Epoch: 46/250 Train Loss: 0.9390 Accuracy: 0.7040 Time: 6.17096  | Val Loss: 0.8469 Accuracy: 0.7391\n",
      "Epoch: 47/250 Train Loss: 0.9127 Accuracy: 0.7086 Time: 6.16118  | Val Loss: 0.8241 Accuracy: 0.7371\n",
      "Epoch: 48/250 Train Loss: 0.8994 Accuracy: 0.7159 Time: 6.09562  | Val Loss: 0.8855 Accuracy: 0.7175\n",
      "Epoch: 49/250 Train Loss: 0.9163 Accuracy: 0.7121 Time: 6.16537  | Val Loss: 0.8907 Accuracy: 0.7164\n",
      "Epoch: 50/250 Train Loss: 0.9014 Accuracy: 0.7110 Time: 6.12856  | Val Loss: 0.8574 Accuracy: 0.7238\n",
      "Epoch: 51/250 Train Loss: 0.9093 Accuracy: 0.7087 Time: 6.12026  | Val Loss: 0.8408 Accuracy: 0.7386\n",
      "Epoch: 52/250 Train Loss: 0.8847 Accuracy: 0.7184 Time: 6.16885  | Val Loss: 0.8467 Accuracy: 0.7332\n",
      "Epoch: 53/250 Train Loss: 0.8751 Accuracy: 0.7249 Time: 6.19262  | Val Loss: 0.9062 Accuracy: 0.7050\n",
      "Epoch: 54/250 Train Loss: 0.8715 Accuracy: 0.7194 Time: 6.19259  | Val Loss: 0.8838 Accuracy: 0.7213\n",
      "Epoch: 55/250 Train Loss: 0.8688 Accuracy: 0.7239 Time: 6.15432  | Val Loss: 0.7871 Accuracy: 0.7521\n",
      "Epoch: 56/250 Train Loss: 0.8680 Accuracy: 0.7224 Time: 6.15209  | Val Loss: 0.7786 Accuracy: 0.7536\n",
      "Epoch: 57/250 Train Loss: 0.8450 Accuracy: 0.7299 Time: 6.18161  | Val Loss: 0.7954 Accuracy: 0.7488\n",
      "Epoch: 58/250 Train Loss: 0.8185 Accuracy: 0.7414 Time: 6.20704  | Val Loss: 0.8587 Accuracy: 0.7246\n",
      "Epoch: 59/250 Train Loss: 0.8362 Accuracy: 0.7349 Time: 6.16318  | Val Loss: 0.7670 Accuracy: 0.7541\n",
      "Epoch: 60/250 Train Loss: 0.8421 Accuracy: 0.7310 Time: 6.14955  | Val Loss: 0.8717 Accuracy: 0.7274\n",
      "Epoch: 61/250 Train Loss: 0.8269 Accuracy: 0.7365 Time: 6.13846  | Val Loss: 0.7814 Accuracy: 0.7498\n",
      "Epoch: 62/250 Train Loss: 0.8533 Accuracy: 0.7227 Time: 6.13787  | Val Loss: 0.8058 Accuracy: 0.7411\n",
      "Epoch: 63/250 Train Loss: 0.8173 Accuracy: 0.7402 Time: 6.11654  | Val Loss: 0.7383 Accuracy: 0.7666\n",
      "Epoch: 64/250 Train Loss: 0.8116 Accuracy: 0.7477 Time: 6.18356  | Val Loss: 0.7630 Accuracy: 0.7564\n",
      "Epoch: 65/250 Train Loss: 0.8083 Accuracy: 0.7457 Time: 6.11593  | Val Loss: 0.8035 Accuracy: 0.7526\n",
      "Epoch: 66/250 Train Loss: 0.8058 Accuracy: 0.7435 Time: 6.15551  | Val Loss: 0.8283 Accuracy: 0.7358\n",
      "Epoch: 67/250 Train Loss: 0.8018 Accuracy: 0.7482 Time: 6.17321  | Val Loss: 0.7777 Accuracy: 0.7544\n",
      "Epoch: 68/250 Train Loss: 0.7832 Accuracy: 0.7567 Time: 6.19091  | Val Loss: 0.7681 Accuracy: 0.7557\n",
      "Epoch: 69/250 Train Loss: 0.7832 Accuracy: 0.7482 Time: 6.14357  | Val Loss: 0.7482 Accuracy: 0.7618\n",
      "Epoch: 70/250 Train Loss: 0.7794 Accuracy: 0.7522 Time: 6.17931  | Val Loss: 0.7153 Accuracy: 0.7776\n",
      "Epoch: 71/250 Train Loss: 0.7759 Accuracy: 0.7547 Time: 6.19840  | Val Loss: 0.7137 Accuracy: 0.7730\n",
      "Epoch: 72/250 Train Loss: 0.7576 Accuracy: 0.7591 Time: 6.14427  | Val Loss: 0.7918 Accuracy: 0.7460\n",
      "Epoch: 73/250 Train Loss: 0.7701 Accuracy: 0.7557 Time: 6.16219  | Val Loss: 0.7228 Accuracy: 0.7722\n",
      "Epoch: 74/250 Train Loss: 0.7557 Accuracy: 0.7575 Time: 6.14511  | Val Loss: 0.8009 Accuracy: 0.7468\n",
      "Epoch: 75/250 Train Loss: 0.7391 Accuracy: 0.7709 Time: 6.14037  | Val Loss: 0.7651 Accuracy: 0.7664\n",
      "Epoch: 76/250 Train Loss: 0.7446 Accuracy: 0.7650 Time: 6.16846  | Val Loss: 0.7104 Accuracy: 0.7804\n",
      "Epoch: 77/250 Train Loss: 0.7497 Accuracy: 0.7643 Time: 6.16061  | Val Loss: 0.7171 Accuracy: 0.7768\n",
      "Epoch: 78/250 Train Loss: 0.7509 Accuracy: 0.7633 Time: 6.13114  | Val Loss: 0.7475 Accuracy: 0.7692\n",
      "Epoch: 79/250 Train Loss: 0.7530 Accuracy: 0.7587 Time: 6.17010  | Val Loss: 0.7400 Accuracy: 0.7689\n",
      "Epoch: 80/250 Train Loss: 0.7438 Accuracy: 0.7639 Time: 6.20849  | Val Loss: 0.7557 Accuracy: 0.7674\n",
      "Epoch: 81/250 Train Loss: 0.7396 Accuracy: 0.7622 Time: 6.19455  | Val Loss: 0.6736 Accuracy: 0.7880\n",
      "Epoch: 82/250 Train Loss: 0.7315 Accuracy: 0.7663 Time: 6.18256  | Val Loss: 0.7503 Accuracy: 0.7689\n",
      "Epoch: 83/250 Train Loss: 0.7190 Accuracy: 0.7671 Time: 6.12663  | Val Loss: 0.7015 Accuracy: 0.7776\n",
      "Epoch: 84/250 Train Loss: 0.7209 Accuracy: 0.7730 Time: 6.19754  | Val Loss: 0.7004 Accuracy: 0.7817\n",
      "Epoch: 85/250 Train Loss: 0.7050 Accuracy: 0.7733 Time: 6.17139  | Val Loss: 0.7545 Accuracy: 0.7631\n",
      "Epoch: 86/250 Train Loss: 0.7055 Accuracy: 0.7759 Time: 6.13644  | Val Loss: 0.7352 Accuracy: 0.7707\n",
      "Epoch: 87/250 Train Loss: 0.7139 Accuracy: 0.7744 Time: 6.11858  | Val Loss: 0.6819 Accuracy: 0.7847\n",
      "Epoch: 88/250 Train Loss: 0.7005 Accuracy: 0.7762 Time: 6.14557  | Val Loss: 0.6467 Accuracy: 0.7941\n",
      "Epoch: 89/250 Train Loss: 0.7010 Accuracy: 0.7785 Time: 6.17807  | Val Loss: 0.7067 Accuracy: 0.7773\n",
      "Epoch: 90/250 Train Loss: 0.6892 Accuracy: 0.7823 Time: 6.17827  | Val Loss: 0.6949 Accuracy: 0.7827\n",
      "Epoch: 91/250 Train Loss: 0.6912 Accuracy: 0.7812 Time: 6.12218  | Val Loss: 0.6952 Accuracy: 0.7883\n",
      "Epoch: 92/250 Train Loss: 0.6885 Accuracy: 0.7793 Time: 6.17612  | Val Loss: 0.6617 Accuracy: 0.7957\n",
      "Epoch: 93/250 Train Loss: 0.6866 Accuracy: 0.7820 Time: 6.15753  | Val Loss: 0.6422 Accuracy: 0.7952\n",
      "Epoch: 94/250 Train Loss: 0.6763 Accuracy: 0.7871 Time: 6.14303  | Val Loss: 0.7012 Accuracy: 0.7852\n",
      "Epoch: 95/250 Train Loss: 0.6690 Accuracy: 0.7905 Time: 6.14544  | Val Loss: 0.6825 Accuracy: 0.7829\n",
      "Epoch: 96/250 Train Loss: 0.6927 Accuracy: 0.7803 Time: 6.15096  | Val Loss: 0.7161 Accuracy: 0.7824\n",
      "Epoch: 97/250 Train Loss: 0.6894 Accuracy: 0.7830 Time: 6.16644  | Val Loss: 0.7062 Accuracy: 0.7832\n",
      "Epoch: 98/250 Train Loss: 0.6583 Accuracy: 0.7899 Time: 6.19519  | Val Loss: 0.6828 Accuracy: 0.7850\n",
      "Epoch: 99/250 Train Loss: 0.6795 Accuracy: 0.7864 Time: 6.16287  | Val Loss: 0.6568 Accuracy: 0.7992\n",
      "Epoch: 100/250 Train Loss: 0.6692 Accuracy: 0.7877 Time: 6.15686  | Val Loss: 0.7175 Accuracy: 0.7778\n",
      "Epoch: 101/250 Train Loss: 0.6523 Accuracy: 0.7906 Time: 6.12105  | Val Loss: 0.6975 Accuracy: 0.7857\n",
      "Epoch: 102/250 Train Loss: 0.6598 Accuracy: 0.7903 Time: 6.16959  | Val Loss: 0.6940 Accuracy: 0.7868\n",
      "Epoch: 103/250 Train Loss: 0.6405 Accuracy: 0.7970 Time: 6.16373  | Val Loss: 0.6554 Accuracy: 0.7924\n",
      "Epoch: 104/250 Train Loss: 0.6566 Accuracy: 0.7893 Time: 6.18936  | Val Loss: 0.6834 Accuracy: 0.7845\n",
      "Epoch: 105/250 Train Loss: 0.6435 Accuracy: 0.7959 Time: 6.24976  | Val Loss: 0.6621 Accuracy: 0.7975\n",
      "Epoch: 106/250 Train Loss: 0.6344 Accuracy: 0.7995 Time: 6.11307  | Val Loss: 0.6627 Accuracy: 0.7946\n",
      "Epoch: 107/250 Train Loss: 0.6332 Accuracy: 0.8003 Time: 6.18533  | Val Loss: 0.7222 Accuracy: 0.7839\n",
      "Epoch: 108/250 Train Loss: 0.6421 Accuracy: 0.7936 Time: 6.17110  | Val Loss: 0.6892 Accuracy: 0.7906\n",
      "Epoch: 109/250 Train Loss: 0.6185 Accuracy: 0.8065 Time: 6.19173  | Val Loss: 0.6757 Accuracy: 0.7903\n",
      "Epoch: 110/250 Train Loss: 0.6382 Accuracy: 0.7961 Time: 6.15037  | Val Loss: 0.6141 Accuracy: 0.8117\n",
      "Epoch: 111/250 Train Loss: 0.6124 Accuracy: 0.8045 Time: 6.15307  | Val Loss: 0.6543 Accuracy: 0.7964\n",
      "Epoch: 112/250 Train Loss: 0.6428 Accuracy: 0.7960 Time: 6.11555  | Val Loss: 0.6307 Accuracy: 0.8061\n",
      "Epoch: 113/250 Train Loss: 0.6419 Accuracy: 0.7948 Time: 6.14089  | Val Loss: 0.6573 Accuracy: 0.8015\n",
      "Epoch: 114/250 Train Loss: 0.6345 Accuracy: 0.7984 Time: 6.12764  | Val Loss: 0.6335 Accuracy: 0.8005\n",
      "Epoch: 115/250 Train Loss: 0.6179 Accuracy: 0.8009 Time: 6.21936  | Val Loss: 0.6533 Accuracy: 0.7962\n",
      "Epoch: 116/250 Train Loss: 0.6154 Accuracy: 0.8040 Time: 6.18418  | Val Loss: 0.6073 Accuracy: 0.8089\n",
      "Epoch: 117/250 Train Loss: 0.6120 Accuracy: 0.8058 Time: 6.15632  | Val Loss: 0.6418 Accuracy: 0.8033\n",
      "Epoch: 118/250 Train Loss: 0.6122 Accuracy: 0.8016 Time: 6.17176  | Val Loss: 0.6415 Accuracy: 0.8008\n",
      "Epoch: 119/250 Train Loss: 0.6010 Accuracy: 0.8138 Time: 6.16267  | Val Loss: 0.6594 Accuracy: 0.8008\n",
      "Epoch: 120/250 Train Loss: 0.5942 Accuracy: 0.8134 Time: 6.14548  | Val Loss: 0.6169 Accuracy: 0.8122\n",
      "Epoch: 121/250 Train Loss: 0.5929 Accuracy: 0.8120 Time: 6.16320  | Val Loss: 0.6961 Accuracy: 0.7901\n",
      "Epoch: 122/250 Train Loss: 0.5874 Accuracy: 0.8098 Time: 6.17355  | Val Loss: 0.6622 Accuracy: 0.8008\n",
      "Epoch: 123/250 Train Loss: 0.5856 Accuracy: 0.8154 Time: 6.18527  | Val Loss: 0.6281 Accuracy: 0.8018\n",
      "Epoch: 124/250 Train Loss: 0.5834 Accuracy: 0.8127 Time: 6.18546  | Val Loss: 0.6506 Accuracy: 0.7982\n",
      "Epoch: 125/250 Train Loss: 0.5659 Accuracy: 0.8215 Time: 6.12565  | Val Loss: 0.6318 Accuracy: 0.8094\n",
      "Epoch: 126/250 Train Loss: 0.5698 Accuracy: 0.8185 Time: 6.17228  | Val Loss: 0.6304 Accuracy: 0.8069\n",
      "Epoch: 127/250 Train Loss: 0.5741 Accuracy: 0.8170 Time: 6.16560  | Val Loss: 0.6290 Accuracy: 0.8069\n",
      "Epoch: 128/250 Train Loss: 0.5553 Accuracy: 0.8263 Time: 6.11554  | Val Loss: 0.6188 Accuracy: 0.8117\n",
      "Epoch: 129/250 Train Loss: 0.5595 Accuracy: 0.8249 Time: 6.13035  | Val Loss: 0.6109 Accuracy: 0.8112\n",
      "Epoch: 130/250 Train Loss: 0.5811 Accuracy: 0.8175 Time: 6.14356  | Val Loss: 0.6233 Accuracy: 0.8092\n",
      "Epoch: 131/250 Train Loss: 0.5419 Accuracy: 0.8279 Time: 6.23102  | Val Loss: 0.6486 Accuracy: 0.8043\n",
      "Epoch: 132/250 Train Loss: 0.5588 Accuracy: 0.8196 Time: 6.17738  | Val Loss: 0.6146 Accuracy: 0.8087\n",
      "Epoch: 133/250 Train Loss: 0.5631 Accuracy: 0.8195 Time: 6.17180  | Val Loss: 0.6054 Accuracy: 0.8130\n",
      "Epoch: 134/250 Train Loss: 0.5510 Accuracy: 0.8236 Time: 6.15201  | Val Loss: 0.6312 Accuracy: 0.8076\n",
      "Epoch: 135/250 Train Loss: 0.5567 Accuracy: 0.8222 Time: 6.14054  | Val Loss: 0.6225 Accuracy: 0.8064\n",
      "Epoch: 136/250 Train Loss: 0.5502 Accuracy: 0.8249 Time: 6.17752  | Val Loss: 0.6329 Accuracy: 0.8069\n",
      "Epoch: 137/250 Train Loss: 0.5550 Accuracy: 0.8237 Time: 6.17328  | Val Loss: 0.6337 Accuracy: 0.8023\n",
      "Epoch: 138/250 Train Loss: 0.5535 Accuracy: 0.8228 Time: 6.20216  | Val Loss: 0.6366 Accuracy: 0.7962\n",
      "Epoch: 139/250 Train Loss: 0.5633 Accuracy: 0.8189 Time: 6.15081  | Val Loss: 0.5911 Accuracy: 0.8171\n",
      "Epoch: 140/250 Train Loss: 0.5318 Accuracy: 0.8322 Time: 6.11003  | Val Loss: 0.5834 Accuracy: 0.8183\n",
      "Epoch: 141/250 Train Loss: 0.5669 Accuracy: 0.8171 Time: 6.17375  | Val Loss: 0.6236 Accuracy: 0.8107\n",
      "Epoch: 142/250 Train Loss: 0.5387 Accuracy: 0.8305 Time: 6.17259  | Val Loss: 0.6081 Accuracy: 0.8214\n",
      "Epoch: 143/250 Train Loss: 0.5285 Accuracy: 0.8355 Time: 6.18235  | Val Loss: 0.6423 Accuracy: 0.8059\n",
      "Epoch: 144/250 Train Loss: 0.5339 Accuracy: 0.8268 Time: 6.12076  | Val Loss: 0.6107 Accuracy: 0.8158\n",
      "Epoch: 145/250 Train Loss: 0.5156 Accuracy: 0.8380 Time: 6.16112  | Val Loss: 0.5924 Accuracy: 0.8206\n",
      "Epoch: 146/250 Train Loss: 0.5299 Accuracy: 0.8329 Time: 6.18143  | Val Loss: 0.6038 Accuracy: 0.8127\n",
      "Epoch: 147/250 Train Loss: 0.5197 Accuracy: 0.8350 Time: 6.18630  | Val Loss: 0.6194 Accuracy: 0.8138\n",
      "Epoch: 148/250 Train Loss: 0.5297 Accuracy: 0.8297 Time: 6.18259  | Val Loss: 0.6016 Accuracy: 0.8191\n",
      "Epoch: 149/250 Train Loss: 0.5356 Accuracy: 0.8348 Time: 6.20436  | Val Loss: 0.5829 Accuracy: 0.8224\n",
      "Epoch: 150/250 Train Loss: 0.5241 Accuracy: 0.8332 Time: 6.30018  | Val Loss: 0.5987 Accuracy: 0.8229\n",
      "Epoch: 151/250 Train Loss: 0.5041 Accuracy: 0.8413 Time: 6.19141  | Val Loss: 0.6020 Accuracy: 0.8186\n",
      "Epoch: 152/250 Train Loss: 0.5092 Accuracy: 0.8412 Time: 6.18792  | Val Loss: 0.5786 Accuracy: 0.8257\n",
      "Epoch: 153/250 Train Loss: 0.5078 Accuracy: 0.8387 Time: 6.17509  | Val Loss: 0.6484 Accuracy: 0.8031\n",
      "Epoch: 154/250 Train Loss: 0.5191 Accuracy: 0.8340 Time: 6.15757  | Val Loss: 0.5876 Accuracy: 0.8234\n",
      "Epoch: 155/250 Train Loss: 0.5141 Accuracy: 0.8391 Time: 6.15828  | Val Loss: 0.6034 Accuracy: 0.8194\n",
      "Epoch: 156/250 Train Loss: 0.5066 Accuracy: 0.8391 Time: 6.16031  | Val Loss: 0.5824 Accuracy: 0.8260\n",
      "Epoch: 157/250 Train Loss: 0.4883 Accuracy: 0.8421 Time: 6.16360  | Val Loss: 0.5938 Accuracy: 0.8194\n",
      "Epoch: 158/250 Train Loss: 0.5191 Accuracy: 0.8351 Time: 6.24212  | Val Loss: 0.6017 Accuracy: 0.8201\n",
      "Epoch: 159/250 Train Loss: 0.5016 Accuracy: 0.8421 Time: 6.13693  | Val Loss: 0.6168 Accuracy: 0.8153\n",
      "Epoch: 160/250 Train Loss: 0.4915 Accuracy: 0.8450 Time: 6.11738  | Val Loss: 0.5782 Accuracy: 0.8194\n",
      "Epoch: 161/250 Train Loss: 0.5073 Accuracy: 0.8416 Time: 6.16596  | Val Loss: 0.6000 Accuracy: 0.8232\n",
      "Epoch: 162/250 Train Loss: 0.4968 Accuracy: 0.8443 Time: 6.16573  | Val Loss: 0.6079 Accuracy: 0.8158\n",
      "Epoch: 163/250 Train Loss: 0.4795 Accuracy: 0.8453 Time: 6.16336  | Val Loss: 0.5697 Accuracy: 0.8283\n",
      "Epoch: 164/250 Train Loss: 0.4933 Accuracy: 0.8450 Time: 6.20500  | Val Loss: 0.5720 Accuracy: 0.8265\n",
      "Epoch: 165/250 Train Loss: 0.4971 Accuracy: 0.8420 Time: 6.18641  | Val Loss: 0.6018 Accuracy: 0.8260\n",
      "Epoch: 166/250 Train Loss: 0.4967 Accuracy: 0.8466 Time: 6.15103  | Val Loss: 0.5857 Accuracy: 0.8239\n",
      "Epoch: 167/250 Train Loss: 0.4913 Accuracy: 0.8422 Time: 6.18958  | Val Loss: 0.5818 Accuracy: 0.8280\n",
      "Epoch: 168/250 Train Loss: 0.4801 Accuracy: 0.8478 Time: 6.18742  | Val Loss: 0.5932 Accuracy: 0.8273\n",
      "Epoch: 169/250 Train Loss: 0.4704 Accuracy: 0.8492 Time: 6.16793  | Val Loss: 0.6020 Accuracy: 0.8204\n",
      "Epoch: 170/250 Train Loss: 0.4852 Accuracy: 0.8489 Time: 6.19910  | Val Loss: 0.5857 Accuracy: 0.8242\n",
      "Epoch: 171/250 Train Loss: 0.4758 Accuracy: 0.8492 Time: 6.15291  | Val Loss: 0.5737 Accuracy: 0.8329\n",
      "Epoch: 172/250 Train Loss: 0.4738 Accuracy: 0.8504 Time: 6.13242  | Val Loss: 0.5885 Accuracy: 0.8209\n",
      "Epoch: 173/250 Train Loss: 0.4731 Accuracy: 0.8459 Time: 6.15329  | Val Loss: 0.5823 Accuracy: 0.8306\n",
      "Epoch: 174/250 Train Loss: 0.4655 Accuracy: 0.8554 Time: 6.17084  | Val Loss: 0.5983 Accuracy: 0.8250\n",
      "Epoch: 175/250 Train Loss: 0.4551 Accuracy: 0.8493 Time: 6.15834  | Val Loss: 0.5866 Accuracy: 0.8257\n",
      "Epoch: 176/250 Train Loss: 0.4586 Accuracy: 0.8584 Time: 6.12935  | Val Loss: 0.6042 Accuracy: 0.8209\n",
      "Epoch: 177/250 Train Loss: 0.4610 Accuracy: 0.8576 Time: 6.07289  | Val Loss: 0.5812 Accuracy: 0.8280\n",
      "Epoch: 178/250 Train Loss: 0.4576 Accuracy: 0.8554 Time: 6.13409  | Val Loss: 0.6102 Accuracy: 0.8201\n",
      "Epoch: 179/250 Train Loss: 0.4686 Accuracy: 0.8523 Time: 6.14864  | Val Loss: 0.5680 Accuracy: 0.8316\n",
      "Epoch: 180/250 Train Loss: 0.4417 Accuracy: 0.8587 Time: 6.12353  | Val Loss: 0.5921 Accuracy: 0.8217\n",
      "Epoch: 181/250 Train Loss: 0.4514 Accuracy: 0.8546 Time: 6.15903  | Val Loss: 0.6067 Accuracy: 0.8232\n",
      "Epoch: 182/250 Train Loss: 0.4369 Accuracy: 0.8627 Time: 6.16393  | Val Loss: 0.5907 Accuracy: 0.8298\n",
      "Epoch: 183/250 Train Loss: 0.4544 Accuracy: 0.8535 Time: 6.15574  | Val Loss: 0.5817 Accuracy: 0.8280\n",
      "Epoch: 184/250 Train Loss: 0.4527 Accuracy: 0.8583 Time: 6.17634  | Val Loss: 0.5792 Accuracy: 0.8268\n",
      "Epoch: 185/250 Train Loss: 0.4452 Accuracy: 0.8607 Time: 6.16913  | Val Loss: 0.5847 Accuracy: 0.8288\n",
      "Epoch: 186/250 Train Loss: 0.4378 Accuracy: 0.8590 Time: 6.17406  | Val Loss: 0.6103 Accuracy: 0.8296\n",
      "Epoch: 187/250 Train Loss: 0.4462 Accuracy: 0.8576 Time: 6.20185  | Val Loss: 0.5947 Accuracy: 0.8270\n",
      "Epoch: 188/250 Train Loss: 0.4482 Accuracy: 0.8576 Time: 6.14472  | Val Loss: 0.5651 Accuracy: 0.8346\n",
      "Epoch: 189/250 Train Loss: 0.4505 Accuracy: 0.8601 Time: 6.14227  | Val Loss: 0.5842 Accuracy: 0.8242\n",
      "Epoch: 190/250 Train Loss: 0.4408 Accuracy: 0.8623 Time: 6.12533  | Val Loss: 0.5752 Accuracy: 0.8336\n",
      "Epoch: 191/250 Train Loss: 0.4262 Accuracy: 0.8639 Time: 6.12623  | Val Loss: 0.5731 Accuracy: 0.8362\n",
      "Epoch: 192/250 Train Loss: 0.4338 Accuracy: 0.8589 Time: 6.15288  | Val Loss: 0.6300 Accuracy: 0.8186\n",
      "Epoch: 193/250 Train Loss: 0.4388 Accuracy: 0.8576 Time: 6.15721  | Val Loss: 0.5956 Accuracy: 0.8283\n",
      "Epoch: 194/250 Train Loss: 0.4320 Accuracy: 0.8634 Time: 6.11943  | Val Loss: 0.5849 Accuracy: 0.8311\n",
      "Epoch: 195/250 Train Loss: 0.4432 Accuracy: 0.8575 Time: 6.14228  | Val Loss: 0.5867 Accuracy: 0.8270\n",
      "Epoch: 196/250 Train Loss: 0.4189 Accuracy: 0.8663 Time: 6.17017  | Val Loss: 0.5894 Accuracy: 0.8237\n",
      "Epoch: 197/250 Train Loss: 0.4229 Accuracy: 0.8638 Time: 6.21342  | Val Loss: 0.5751 Accuracy: 0.8359\n",
      "Epoch: 198/250 Train Loss: 0.4369 Accuracy: 0.8629 Time: 6.16232  | Val Loss: 0.5735 Accuracy: 0.8311\n",
      "Epoch: 199/250 Train Loss: 0.4231 Accuracy: 0.8677 Time: 6.18240  | Val Loss: 0.5764 Accuracy: 0.8324\n",
      "Epoch: 200/250 Train Loss: 0.4354 Accuracy: 0.8624 Time: 6.12866  | Val Loss: 0.5909 Accuracy: 0.8321\n",
      "Epoch: 201/250 Train Loss: 0.4316 Accuracy: 0.8634 Time: 6.20291  | Val Loss: 0.5935 Accuracy: 0.8242\n",
      "Epoch: 202/250 Train Loss: 0.4254 Accuracy: 0.8660 Time: 6.18511  | Val Loss: 0.5747 Accuracy: 0.8331\n",
      "Epoch: 203/250 Train Loss: 0.4244 Accuracy: 0.8648 Time: 6.15616  | Val Loss: 0.6229 Accuracy: 0.8237\n",
      "Epoch: 204/250 Train Loss: 0.4213 Accuracy: 0.8679 Time: 6.21691  | Val Loss: 0.5777 Accuracy: 0.8306\n",
      "Epoch: 205/250 Train Loss: 0.4231 Accuracy: 0.8618 Time: 6.16824  | Val Loss: 0.5752 Accuracy: 0.8331\n",
      "Epoch: 206/250 Train Loss: 0.4143 Accuracy: 0.8698 Time: 6.14280  | Val Loss: 0.5683 Accuracy: 0.8326\n",
      "Epoch: 207/250 Train Loss: 0.4204 Accuracy: 0.8662 Time: 6.12063  | Val Loss: 0.5826 Accuracy: 0.8369\n",
      "Epoch: 208/250 Train Loss: 0.4049 Accuracy: 0.8685 Time: 6.16567  | Val Loss: 0.5859 Accuracy: 0.8321\n",
      "Epoch: 209/250 Train Loss: 0.4127 Accuracy: 0.8681 Time: 6.12481  | Val Loss: 0.5736 Accuracy: 0.8375\n",
      "Epoch: 210/250 Train Loss: 0.4051 Accuracy: 0.8736 Time: 6.18040  | Val Loss: 0.5815 Accuracy: 0.8336\n",
      "Epoch: 211/250 Train Loss: 0.4076 Accuracy: 0.8707 Time: 6.13305  | Val Loss: 0.5726 Accuracy: 0.8367\n",
      "Epoch: 212/250 Train Loss: 0.4077 Accuracy: 0.8694 Time: 6.16483  | Val Loss: 0.5745 Accuracy: 0.8362\n",
      "Epoch: 213/250 Train Loss: 0.4045 Accuracy: 0.8730 Time: 6.13985  | Val Loss: 0.5879 Accuracy: 0.8313\n",
      "Epoch: 214/250 Train Loss: 0.4101 Accuracy: 0.8705 Time: 6.13399  | Val Loss: 0.5845 Accuracy: 0.8346\n",
      "Epoch: 215/250 Train Loss: 0.4089 Accuracy: 0.8680 Time: 6.12197  | Val Loss: 0.5760 Accuracy: 0.8380\n",
      "Epoch: 216/250 Train Loss: 0.4179 Accuracy: 0.8693 Time: 6.14220  | Val Loss: 0.5759 Accuracy: 0.8341\n",
      "Epoch: 217/250 Train Loss: 0.4025 Accuracy: 0.8774 Time: 6.15700  | Val Loss: 0.5630 Accuracy: 0.8392\n",
      "Epoch: 218/250 Train Loss: 0.3990 Accuracy: 0.8765 Time: 6.16978  | Val Loss: 0.5729 Accuracy: 0.8385\n",
      "Epoch: 219/250 Train Loss: 0.4075 Accuracy: 0.8723 Time: 6.14964  | Val Loss: 0.5732 Accuracy: 0.8359\n",
      "Epoch: 220/250 Train Loss: 0.4107 Accuracy: 0.8702 Time: 6.16688  | Val Loss: 0.5726 Accuracy: 0.8387\n",
      "Epoch: 221/250 Train Loss: 0.4098 Accuracy: 0.8744 Time: 6.12286  | Val Loss: 0.5702 Accuracy: 0.8385\n",
      "Epoch: 222/250 Train Loss: 0.4038 Accuracy: 0.8776 Time: 6.22644  | Val Loss: 0.5694 Accuracy: 0.8362\n",
      "Epoch: 223/250 Train Loss: 0.3920 Accuracy: 0.8767 Time: 6.10517  | Val Loss: 0.5731 Accuracy: 0.8344\n",
      "Epoch: 224/250 Train Loss: 0.3952 Accuracy: 0.8777 Time: 6.13520  | Val Loss: 0.5700 Accuracy: 0.8364\n",
      "Epoch: 225/250 Train Loss: 0.3951 Accuracy: 0.8758 Time: 6.17565  | Val Loss: 0.5771 Accuracy: 0.8352\n",
      "Epoch: 226/250 Train Loss: 0.4015 Accuracy: 0.8739 Time: 6.10999  | Val Loss: 0.5706 Accuracy: 0.8372\n",
      "Epoch: 227/250 Train Loss: 0.3942 Accuracy: 0.8744 Time: 6.15640  | Val Loss: 0.5677 Accuracy: 0.8390\n",
      "Epoch: 228/250 Train Loss: 0.3943 Accuracy: 0.8750 Time: 6.14574  | Val Loss: 0.5770 Accuracy: 0.8377\n",
      "Epoch: 229/250 Train Loss: 0.3988 Accuracy: 0.8719 Time: 6.16307  | Val Loss: 0.5808 Accuracy: 0.8349\n",
      "Epoch: 230/250 Train Loss: 0.3934 Accuracy: 0.8775 Time: 6.15290  | Val Loss: 0.5707 Accuracy: 0.8377\n",
      "Epoch: 231/250 Train Loss: 0.4129 Accuracy: 0.8725 Time: 6.13502  | Val Loss: 0.5703 Accuracy: 0.8380\n",
      "Epoch: 232/250 Train Loss: 0.3884 Accuracy: 0.8732 Time: 6.21280  | Val Loss: 0.5711 Accuracy: 0.8397\n",
      "Epoch: 233/250 Train Loss: 0.3817 Accuracy: 0.8765 Time: 6.15117  | Val Loss: 0.5785 Accuracy: 0.8390\n",
      "Epoch: 234/250 Train Loss: 0.3895 Accuracy: 0.8758 Time: 6.16121  | Val Loss: 0.5745 Accuracy: 0.8387\n",
      "Epoch: 235/250 Train Loss: 0.3863 Accuracy: 0.8825 Time: 6.18235  | Val Loss: 0.5734 Accuracy: 0.8395\n",
      "Epoch: 236/250 Train Loss: 0.4075 Accuracy: 0.8718 Time: 6.06031  | Val Loss: 0.5691 Accuracy: 0.8413\n",
      "Epoch: 237/250 Train Loss: 0.3886 Accuracy: 0.8777 Time: 6.13624  | Val Loss: 0.5694 Accuracy: 0.8423\n",
      "Epoch: 238/250 Train Loss: 0.4033 Accuracy: 0.8747 Time: 6.13967  | Val Loss: 0.5668 Accuracy: 0.8403\n",
      "Epoch: 239/250 Train Loss: 0.3908 Accuracy: 0.8717 Time: 6.20383  | Val Loss: 0.5663 Accuracy: 0.8400\n",
      "Epoch: 240/250 Train Loss: 0.3973 Accuracy: 0.8728 Time: 6.11418  | Val Loss: 0.5747 Accuracy: 0.8369\n",
      "Epoch: 241/250 Train Loss: 0.3910 Accuracy: 0.8772 Time: 6.11027  | Val Loss: 0.5671 Accuracy: 0.8390\n",
      "Epoch: 242/250 Train Loss: 0.3891 Accuracy: 0.8762 Time: 6.14730  | Val Loss: 0.5704 Accuracy: 0.8387\n",
      "Epoch: 243/250 Train Loss: 0.3831 Accuracy: 0.8800 Time: 6.13767  | Val Loss: 0.5707 Accuracy: 0.8408\n",
      "Epoch: 244/250 Train Loss: 0.3958 Accuracy: 0.8756 Time: 6.19697  | Val Loss: 0.5721 Accuracy: 0.8390\n",
      "Epoch: 245/250 Train Loss: 0.3879 Accuracy: 0.8781 Time: 6.16015  | Val Loss: 0.5692 Accuracy: 0.8408\n",
      "Epoch: 246/250 Train Loss: 0.4035 Accuracy: 0.8745 Time: 6.17347  | Val Loss: 0.5651 Accuracy: 0.8403\n",
      "Epoch: 247/250 Train Loss: 0.4058 Accuracy: 0.8722 Time: 6.19365  | Val Loss: 0.5689 Accuracy: 0.8397\n",
      "Epoch: 248/250 Train Loss: 0.3881 Accuracy: 0.8774 Time: 6.13490  | Val Loss: 0.5684 Accuracy: 0.8408\n",
      "Epoch: 249/250 Train Loss: 0.4009 Accuracy: 0.8727 Time: 6.16025  | Val Loss: 0.5666 Accuracy: 0.8395\n",
      "Epoch: 250/250 Train Loss: 0.3854 Accuracy: 0.8745 Time: 6.18559  | Val Loss: 0.5698 Accuracy: 0.8382\n",
      "#Parameter: 727626 Accuracy: 0.8384713375796178\n"
     ]
    }
   ],
   "source": [
    "from typing import Tuple\n",
    "from hdd.models.cnn.squeezenet import SqueezeNet\n",
    "from hdd.train.classification_utils import (\n",
    "    naive_train_classification_model,\n",
    "    eval_image_classifier,\n",
    "    _train_classifier_naive,\n",
    ")\n",
    "from hdd.models.nn_utils import count_trainable_parameter\n",
    "\n",
    "\n",
    "def train_net(\n",
    "    train_dataloader,\n",
    "    val_dataloader,\n",
    "    add_norm,\n",
    "    dropout,\n",
    "    lr=1e-3,\n",
    "    weight_decay=1e-5,\n",
    "    max_epochs=150,\n",
    "    train_classifier=None,\n",
    ") -> tuple[SqueezeNet, dict[str, list[float]]]:\n",
    "    net = SqueezeNet(num_classes=10, add_norm=add_norm, dropout=dropout).to(DEVICE)\n",
    "    print(f\"#Parameter: {count_trainable_parameter(net)}\")\n",
    "    criteria = nn.CrossEntropyLoss()\n",
    "    optimizer = torch.optim.SGD(\n",
    "        net.parameters(), lr=lr, momentum=0.9, weight_decay=weight_decay\n",
    "    )\n",
    "\n",
    "    scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(\n",
    "        optimizer, max_epochs, eta_min=lr / 100\n",
    "    )\n",
    "    if train_classifier is None:\n",
    "        train_classifier = _train_classifier_naive\n",
    "    training_stats = naive_train_classification_model(\n",
    "        net,\n",
    "        criteria,\n",
    "        max_epochs,\n",
    "        train_dataloader,\n",
    "        val_dataloader,\n",
    "        DEVICE,\n",
    "        optimizer,\n",
    "        scheduler,\n",
    "        verbose=True,\n",
    "        train_classifier=train_classifier,\n",
    "    )\n",
    "    return net, training_stats\n",
    "\n",
    "\n",
    "train_dataloader, val_dataloader = build_dataloader(128, train_dataset, val_dataset)\n",
    "\n",
    "\n",
    "def train_classifier_with_gradient_clipping(\n",
    "    net: nn.Module,\n",
    "    criteria: nn.CrossEntropyLoss,\n",
    "    optimizer: optim.Optimizer,\n",
    "    train_loader: torch.utils.data.DataLoader,\n",
    "    device: torch.device,\n",
    ") -> Tuple[float, float]:\n",
    "    \"\"\"Naive training procedure to train classifier for one epoch.\n",
    "\n",
    "    Args:\n",
    "        net: network instance.\n",
    "        criteria: Loss function. Typically nn.CrossEntropyLoss\n",
    "        optimizer: optimizer.\n",
    "        train_loader: train data\n",
    "        device: device to run the training.\n",
    "\n",
    "    Returns:\n",
    "        avg train loss and train accuracy.\n",
    "    \"\"\"\n",
    "\n",
    "    train_loss = 0.0\n",
    "    correct_items = 0\n",
    "    total_items = 0\n",
    "    net.train()\n",
    "    for Xs, ys in train_loader:\n",
    "        Xs, ys = Xs.to(device), ys.to(device)\n",
    "        optimizer.zero_grad()\n",
    "        logits = net(Xs)\n",
    "        loss = criteria(logits, ys)\n",
    "        loss.backward()\n",
    "        torch.nn.utils.clip_grad_norm_(net.parameters(), max_norm=1.0)\n",
    "        optimizer.step()\n",
    "        train_loss += loss.item()\n",
    "        correct_items += torch.sum(torch.argmax(logits, dim=1) == ys).item()\n",
    "        total_items += Xs.shape[0]\n",
    "\n",
    "    avg_train_loss = train_loss / len(train_loader)\n",
    "    accuracy = correct_items / total_items\n",
    "    return avg_train_loss, accuracy\n",
    "\n",
    "\n",
    "# 在不添加batch norm的情况下,这里用了gradient clipping,否则会有梯度爆炸\n",
    "net, no_norm_stats = train_net(\n",
    "    train_dataloader,\n",
    "    val_dataloader,\n",
    "    add_norm=False,\n",
    "    dropout=0,\n",
    "    lr=0.01,\n",
    "    weight_decay=0,\n",
    "    max_epochs=250,\n",
    "    train_classifier=train_classifier_with_gradient_clipping,\n",
    ")\n",
    "\n",
    "eval_result = eval_image_classifier(net, val_dataloader.dataset, DEVICE)\n",
    "ss = [result.gt_label == result.predicted_label for result in eval_result]\n",
    "print(f\"#Parameter: {count_trainable_parameter(net)} Accuracy: {sum(ss) / len(ss)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "faa762f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#Parameter: 733514\n",
      "Epoch: 1/250 Train Loss: 3.0885 Accuracy: 0.2329 Time: 7.63528  | Val Loss: 2.0716 Accuracy: 0.3004\n",
      "Epoch: 2/250 Train Loss: 2.0264 Accuracy: 0.3179 Time: 7.60864  | Val Loss: 1.9494 Accuracy: 0.3549\n",
      "Epoch: 3/250 Train Loss: 1.8853 Accuracy: 0.3781 Time: 7.59596  | Val Loss: 1.8920 Accuracy: 0.3814\n",
      "Epoch: 4/250 Train Loss: 1.7319 Accuracy: 0.4372 Time: 7.57057  | Val Loss: 1.8101 Accuracy: 0.4125\n",
      "Epoch: 5/250 Train Loss: 1.6501 Accuracy: 0.4624 Time: 7.59889  | Val Loss: 1.5483 Accuracy: 0.4922\n",
      "Epoch: 6/250 Train Loss: 1.4677 Accuracy: 0.5215 Time: 7.60585  | Val Loss: 1.4188 Accuracy: 0.5271\n",
      "Epoch: 7/250 Train Loss: 1.3366 Accuracy: 0.5610 Time: 7.57479  | Val Loss: 1.3409 Accuracy: 0.5661\n",
      "Epoch: 8/250 Train Loss: 1.2924 Accuracy: 0.5825 Time: 7.57574  | Val Loss: 1.2044 Accuracy: 0.6140\n",
      "Epoch: 9/250 Train Loss: 1.2035 Accuracy: 0.6147 Time: 7.60885  | Val Loss: 1.3576 Accuracy: 0.5824\n",
      "Epoch: 10/250 Train Loss: 1.1685 Accuracy: 0.6252 Time: 7.58097  | Val Loss: 1.2220 Accuracy: 0.6054\n",
      "Epoch: 11/250 Train Loss: 1.1515 Accuracy: 0.6304 Time: 7.60738  | Val Loss: 1.0540 Accuracy: 0.6683\n",
      "Epoch: 12/250 Train Loss: 1.0907 Accuracy: 0.6493 Time: 7.61822  | Val Loss: 0.9899 Accuracy: 0.6866\n",
      "Epoch: 13/250 Train Loss: 1.0713 Accuracy: 0.6591 Time: 7.64537  | Val Loss: 1.2337 Accuracy: 0.5995\n",
      "Epoch: 14/250 Train Loss: 1.0412 Accuracy: 0.6639 Time: 7.62931  | Val Loss: 1.0336 Accuracy: 0.6639\n",
      "Epoch: 15/250 Train Loss: 1.0195 Accuracy: 0.6738 Time: 7.61553  | Val Loss: 0.9360 Accuracy: 0.6968\n",
      "Epoch: 16/250 Train Loss: 0.9743 Accuracy: 0.6935 Time: 7.58262  | Val Loss: 0.9880 Accuracy: 0.6899\n",
      "Epoch: 17/250 Train Loss: 0.9294 Accuracy: 0.7002 Time: 7.61073  | Val Loss: 1.0233 Accuracy: 0.6869\n",
      "Epoch: 18/250 Train Loss: 0.9354 Accuracy: 0.6999 Time: 7.64961  | Val Loss: 0.8835 Accuracy: 0.7228\n",
      "Epoch: 19/250 Train Loss: 0.9224 Accuracy: 0.7112 Time: 7.61960  | Val Loss: 0.8738 Accuracy: 0.7215\n",
      "Epoch: 20/250 Train Loss: 0.8861 Accuracy: 0.7176 Time: 7.63318  | Val Loss: 0.8610 Accuracy: 0.7315\n",
      "Epoch: 21/250 Train Loss: 0.8521 Accuracy: 0.7275 Time: 7.65258  | Val Loss: 0.8598 Accuracy: 0.7231\n",
      "Epoch: 22/250 Train Loss: 0.8561 Accuracy: 0.7252 Time: 7.61920  | Val Loss: 1.0033 Accuracy: 0.6841\n",
      "Epoch: 23/250 Train Loss: 0.8406 Accuracy: 0.7278 Time: 7.70983  | Val Loss: 0.9066 Accuracy: 0.7154\n",
      "Epoch: 24/250 Train Loss: 0.7981 Accuracy: 0.7444 Time: 7.61249  | Val Loss: 0.8488 Accuracy: 0.7396\n",
      "Epoch: 25/250 Train Loss: 0.8210 Accuracy: 0.7366 Time: 7.69403  | Val Loss: 0.9006 Accuracy: 0.7162\n",
      "Epoch: 26/250 Train Loss: 0.7898 Accuracy: 0.7434 Time: 7.67675  | Val Loss: 0.9224 Accuracy: 0.7205\n",
      "Epoch: 27/250 Train Loss: 0.7773 Accuracy: 0.7537 Time: 7.60063  | Val Loss: 0.8068 Accuracy: 0.7554\n",
      "Epoch: 28/250 Train Loss: 0.7461 Accuracy: 0.7610 Time: 7.57446  | Val Loss: 0.7993 Accuracy: 0.7473\n",
      "Epoch: 29/250 Train Loss: 0.7461 Accuracy: 0.7592 Time: 7.63009  | Val Loss: 0.7815 Accuracy: 0.7432\n",
      "Epoch: 30/250 Train Loss: 0.7270 Accuracy: 0.7676 Time: 7.64494  | Val Loss: 0.7149 Accuracy: 0.7676\n",
      "Epoch: 31/250 Train Loss: 0.7350 Accuracy: 0.7629 Time: 7.60921  | Val Loss: 0.7828 Accuracy: 0.7564\n",
      "Epoch: 32/250 Train Loss: 0.6925 Accuracy: 0.7790 Time: 7.61149  | Val Loss: 0.6735 Accuracy: 0.7868\n",
      "Epoch: 33/250 Train Loss: 0.6985 Accuracy: 0.7818 Time: 7.60732  | Val Loss: 0.6858 Accuracy: 0.7847\n",
      "Epoch: 34/250 Train Loss: 0.6838 Accuracy: 0.7793 Time: 7.63629  | Val Loss: 0.7285 Accuracy: 0.7710\n",
      "Epoch: 35/250 Train Loss: 0.6726 Accuracy: 0.7875 Time: 7.58632  | Val Loss: 0.7094 Accuracy: 0.7738\n",
      "Epoch: 36/250 Train Loss: 0.6592 Accuracy: 0.7920 Time: 7.67164  | Val Loss: 0.6853 Accuracy: 0.7873\n",
      "Epoch: 37/250 Train Loss: 0.6469 Accuracy: 0.7922 Time: 7.60473  | Val Loss: 0.6680 Accuracy: 0.7829\n",
      "Epoch: 38/250 Train Loss: 0.6411 Accuracy: 0.7992 Time: 7.63651  | Val Loss: 0.6809 Accuracy: 0.7878\n",
      "Epoch: 39/250 Train Loss: 0.6469 Accuracy: 0.7930 Time: 7.64027  | Val Loss: 0.7272 Accuracy: 0.7648\n",
      "Epoch: 40/250 Train Loss: 0.6292 Accuracy: 0.7940 Time: 7.64742  | Val Loss: 0.6815 Accuracy: 0.7880\n",
      "Epoch: 41/250 Train Loss: 0.6196 Accuracy: 0.8031 Time: 7.62943  | Val Loss: 0.6245 Accuracy: 0.8015\n",
      "Epoch: 42/250 Train Loss: 0.6287 Accuracy: 0.8006 Time: 7.66685  | Val Loss: 0.6480 Accuracy: 0.7975\n",
      "Epoch: 43/250 Train Loss: 0.6101 Accuracy: 0.8091 Time: 7.60360  | Val Loss: 0.7116 Accuracy: 0.7750\n",
      "Epoch: 44/250 Train Loss: 0.5965 Accuracy: 0.8082 Time: 7.60269  | Val Loss: 0.6822 Accuracy: 0.7893\n",
      "Epoch: 45/250 Train Loss: 0.6125 Accuracy: 0.8036 Time: 7.60678  | Val Loss: 0.6170 Accuracy: 0.8051\n",
      "Epoch: 46/250 Train Loss: 0.5817 Accuracy: 0.8144 Time: 7.65784  | Val Loss: 0.6627 Accuracy: 0.7865\n",
      "Epoch: 47/250 Train Loss: 0.5638 Accuracy: 0.8159 Time: 7.60927  | Val Loss: 0.6924 Accuracy: 0.7824\n",
      "Epoch: 48/250 Train Loss: 0.5553 Accuracy: 0.8180 Time: 7.61186  | Val Loss: 0.6738 Accuracy: 0.7865\n",
      "Epoch: 49/250 Train Loss: 0.5594 Accuracy: 0.8219 Time: 7.64651  | Val Loss: 0.7072 Accuracy: 0.7809\n",
      "Epoch: 50/250 Train Loss: 0.5633 Accuracy: 0.8209 Time: 7.66841  | Val Loss: 0.7150 Accuracy: 0.7888\n",
      "Epoch: 51/250 Train Loss: 0.5522 Accuracy: 0.8221 Time: 7.58788  | Val Loss: 0.6247 Accuracy: 0.8102\n",
      "Epoch: 52/250 Train Loss: 0.5458 Accuracy: 0.8257 Time: 7.59201  | Val Loss: 0.5994 Accuracy: 0.8150\n",
      "Epoch: 53/250 Train Loss: 0.5418 Accuracy: 0.8248 Time: 7.60400  | Val Loss: 0.6203 Accuracy: 0.8138\n",
      "Epoch: 54/250 Train Loss: 0.5232 Accuracy: 0.8314 Time: 7.60437  | Val Loss: 0.6825 Accuracy: 0.7967\n",
      "Epoch: 55/250 Train Loss: 0.5227 Accuracy: 0.8317 Time: 7.62275  | Val Loss: 0.6559 Accuracy: 0.7995\n",
      "Epoch: 56/250 Train Loss: 0.5331 Accuracy: 0.8262 Time: 7.63624  | Val Loss: 0.6326 Accuracy: 0.8028\n",
      "Epoch: 57/250 Train Loss: 0.5230 Accuracy: 0.8328 Time: 7.61244  | Val Loss: 0.7118 Accuracy: 0.7913\n",
      "Epoch: 58/250 Train Loss: 0.5064 Accuracy: 0.8372 Time: 7.60985  | Val Loss: 0.6880 Accuracy: 0.7929\n",
      "Epoch: 59/250 Train Loss: 0.4973 Accuracy: 0.8392 Time: 7.60960  | Val Loss: 0.6024 Accuracy: 0.8196\n",
      "Epoch: 60/250 Train Loss: 0.4901 Accuracy: 0.8419 Time: 7.65929  | Val Loss: 0.5845 Accuracy: 0.8194\n",
      "Epoch: 61/250 Train Loss: 0.4734 Accuracy: 0.8458 Time: 7.59983  | Val Loss: 0.6821 Accuracy: 0.7931\n",
      "Epoch: 62/250 Train Loss: 0.4754 Accuracy: 0.8493 Time: 7.64447  | Val Loss: 0.7746 Accuracy: 0.7855\n",
      "Epoch: 63/250 Train Loss: 0.4753 Accuracy: 0.8466 Time: 7.59499  | Val Loss: 0.7229 Accuracy: 0.7687\n",
      "Epoch: 64/250 Train Loss: 0.4585 Accuracy: 0.8515 Time: 7.66099  | Val Loss: 0.6128 Accuracy: 0.8076\n",
      "Epoch: 65/250 Train Loss: 0.4617 Accuracy: 0.8511 Time: 7.60657  | Val Loss: 0.6840 Accuracy: 0.7949\n",
      "Epoch: 66/250 Train Loss: 0.4594 Accuracy: 0.8471 Time: 7.66276  | Val Loss: 0.7034 Accuracy: 0.7860\n",
      "Epoch: 67/250 Train Loss: 0.4702 Accuracy: 0.8487 Time: 7.60869  | Val Loss: 0.6010 Accuracy: 0.8196\n",
      "Epoch: 68/250 Train Loss: 0.4426 Accuracy: 0.8568 Time: 7.66412  | Val Loss: 0.5689 Accuracy: 0.8265\n",
      "Epoch: 69/250 Train Loss: 0.4376 Accuracy: 0.8566 Time: 7.59429  | Val Loss: 0.5683 Accuracy: 0.8321\n",
      "Epoch: 70/250 Train Loss: 0.4636 Accuracy: 0.8496 Time: 7.63568  | Val Loss: 0.5879 Accuracy: 0.8168\n",
      "Epoch: 71/250 Train Loss: 0.4427 Accuracy: 0.8548 Time: 7.61752  | Val Loss: 0.6406 Accuracy: 0.8117\n",
      "Epoch: 72/250 Train Loss: 0.4207 Accuracy: 0.8655 Time: 7.64783  | Val Loss: 0.5998 Accuracy: 0.8150\n",
      "Epoch: 73/250 Train Loss: 0.4166 Accuracy: 0.8639 Time: 7.57944  | Val Loss: 0.5713 Accuracy: 0.8227\n",
      "Epoch: 74/250 Train Loss: 0.4161 Accuracy: 0.8654 Time: 7.63742  | Val Loss: 0.5427 Accuracy: 0.8308\n",
      "Epoch: 75/250 Train Loss: 0.4238 Accuracy: 0.8623 Time: 7.66441  | Val Loss: 0.5586 Accuracy: 0.8321\n",
      "Epoch: 76/250 Train Loss: 0.4002 Accuracy: 0.8686 Time: 7.61874  | Val Loss: 0.6324 Accuracy: 0.8130\n",
      "Epoch: 77/250 Train Loss: 0.4017 Accuracy: 0.8705 Time: 7.68446  | Val Loss: 0.6546 Accuracy: 0.8038\n",
      "Epoch: 78/250 Train Loss: 0.4068 Accuracy: 0.8677 Time: 7.60630  | Val Loss: 0.5543 Accuracy: 0.8313\n",
      "Epoch: 79/250 Train Loss: 0.3953 Accuracy: 0.8753 Time: 7.59862  | Val Loss: 0.5891 Accuracy: 0.8163\n",
      "Epoch: 80/250 Train Loss: 0.3884 Accuracy: 0.8709 Time: 7.63903  | Val Loss: 0.5566 Accuracy: 0.8232\n",
      "Epoch: 81/250 Train Loss: 0.3717 Accuracy: 0.8813 Time: 7.63016  | Val Loss: 0.6158 Accuracy: 0.8204\n",
      "Epoch: 82/250 Train Loss: 0.3890 Accuracy: 0.8736 Time: 7.62745  | Val Loss: 0.6260 Accuracy: 0.8178\n",
      "Epoch: 83/250 Train Loss: 0.3680 Accuracy: 0.8790 Time: 7.62009  | Val Loss: 0.6251 Accuracy: 0.8148\n",
      "Epoch: 84/250 Train Loss: 0.3990 Accuracy: 0.8734 Time: 7.60469  | Val Loss: 0.6511 Accuracy: 0.8104\n",
      "Epoch: 85/250 Train Loss: 0.3744 Accuracy: 0.8778 Time: 7.58873  | Val Loss: 0.5478 Accuracy: 0.8301\n",
      "Epoch: 86/250 Train Loss: 0.3645 Accuracy: 0.8809 Time: 7.64741  | Val Loss: 0.6699 Accuracy: 0.8102\n",
      "Epoch: 87/250 Train Loss: 0.3663 Accuracy: 0.8803 Time: 7.61131  | Val Loss: 0.6547 Accuracy: 0.8158\n",
      "Epoch: 88/250 Train Loss: 0.3601 Accuracy: 0.8848 Time: 7.58998  | Val Loss: 0.5856 Accuracy: 0.8318\n",
      "Epoch: 89/250 Train Loss: 0.3666 Accuracy: 0.8811 Time: 7.70197  | Val Loss: 0.5362 Accuracy: 0.8403\n",
      "Epoch: 90/250 Train Loss: 0.3500 Accuracy: 0.8843 Time: 7.59008  | Val Loss: 0.6103 Accuracy: 0.8178\n",
      "Epoch: 91/250 Train Loss: 0.3523 Accuracy: 0.8841 Time: 7.64523  | Val Loss: 0.6395 Accuracy: 0.8132\n",
      "Epoch: 92/250 Train Loss: 0.3511 Accuracy: 0.8868 Time: 7.58373  | Val Loss: 0.6054 Accuracy: 0.8186\n",
      "Epoch: 93/250 Train Loss: 0.3548 Accuracy: 0.8867 Time: 7.60908  | Val Loss: 0.5575 Accuracy: 0.8403\n",
      "Epoch: 94/250 Train Loss: 0.3445 Accuracy: 0.8864 Time: 7.58898  | Val Loss: 0.6031 Accuracy: 0.8237\n",
      "Epoch: 95/250 Train Loss: 0.3388 Accuracy: 0.8900 Time: 7.62845  | Val Loss: 0.5952 Accuracy: 0.8364\n",
      "Epoch: 96/250 Train Loss: 0.3341 Accuracy: 0.8913 Time: 7.61826  | Val Loss: 0.6131 Accuracy: 0.8245\n",
      "Epoch: 97/250 Train Loss: 0.3282 Accuracy: 0.8951 Time: 7.59897  | Val Loss: 0.5323 Accuracy: 0.8395\n",
      "Epoch: 98/250 Train Loss: 0.3390 Accuracy: 0.8923 Time: 7.63608  | Val Loss: 0.5984 Accuracy: 0.8275\n",
      "Epoch: 99/250 Train Loss: 0.3210 Accuracy: 0.8962 Time: 7.62718  | Val Loss: 0.6304 Accuracy: 0.8153\n",
      "Epoch: 100/250 Train Loss: 0.3123 Accuracy: 0.8972 Time: 7.59005  | Val Loss: 0.5731 Accuracy: 0.8349\n",
      "Epoch: 101/250 Train Loss: 0.3178 Accuracy: 0.8979 Time: 7.62626  | Val Loss: 0.5197 Accuracy: 0.8441\n",
      "Epoch: 102/250 Train Loss: 0.3138 Accuracy: 0.8976 Time: 7.62954  | Val Loss: 0.5649 Accuracy: 0.8334\n",
      "Epoch: 103/250 Train Loss: 0.3145 Accuracy: 0.8995 Time: 7.61647  | Val Loss: 0.6565 Accuracy: 0.8166\n",
      "Epoch: 104/250 Train Loss: 0.3104 Accuracy: 0.8997 Time: 7.65378  | Val Loss: 0.6171 Accuracy: 0.8183\n",
      "Epoch: 105/250 Train Loss: 0.3195 Accuracy: 0.8973 Time: 7.62080  | Val Loss: 0.5901 Accuracy: 0.8301\n",
      "Epoch: 106/250 Train Loss: 0.2993 Accuracy: 0.9025 Time: 7.65598  | Val Loss: 0.5521 Accuracy: 0.8438\n",
      "Epoch: 107/250 Train Loss: 0.2964 Accuracy: 0.9012 Time: 7.64702  | Val Loss: 0.5957 Accuracy: 0.8324\n",
      "Epoch: 108/250 Train Loss: 0.2954 Accuracy: 0.9078 Time: 7.66132  | Val Loss: 0.6013 Accuracy: 0.8270\n",
      "Epoch: 109/250 Train Loss: 0.2994 Accuracy: 0.8987 Time: 7.64440  | Val Loss: 0.5605 Accuracy: 0.8311\n",
      "Epoch: 110/250 Train Loss: 0.2958 Accuracy: 0.9047 Time: 7.64753  | Val Loss: 0.6258 Accuracy: 0.8324\n",
      "Epoch: 111/250 Train Loss: 0.2903 Accuracy: 0.9076 Time: 7.61716  | Val Loss: 0.6044 Accuracy: 0.8308\n",
      "Epoch: 112/250 Train Loss: 0.2825 Accuracy: 0.9080 Time: 7.61779  | Val Loss: 0.6010 Accuracy: 0.8311\n",
      "Epoch: 113/250 Train Loss: 0.2776 Accuracy: 0.9093 Time: 7.63411  | Val Loss: 0.5840 Accuracy: 0.8448\n",
      "Epoch: 114/250 Train Loss: 0.2659 Accuracy: 0.9120 Time: 7.61632  | Val Loss: 0.6647 Accuracy: 0.8255\n",
      "Epoch: 115/250 Train Loss: 0.2840 Accuracy: 0.9051 Time: 7.64790  | Val Loss: 0.5826 Accuracy: 0.8380\n",
      "Epoch: 116/250 Train Loss: 0.2693 Accuracy: 0.9134 Time: 7.62006  | Val Loss: 0.5763 Accuracy: 0.8367\n",
      "Epoch: 117/250 Train Loss: 0.2683 Accuracy: 0.9136 Time: 7.63336  | Val Loss: 0.5851 Accuracy: 0.8311\n",
      "Epoch: 118/250 Train Loss: 0.2754 Accuracy: 0.9112 Time: 7.62751  | Val Loss: 0.5397 Accuracy: 0.8413\n",
      "Epoch: 119/250 Train Loss: 0.2735 Accuracy: 0.9125 Time: 7.64202  | Val Loss: 0.6342 Accuracy: 0.8199\n",
      "Epoch: 120/250 Train Loss: 0.2711 Accuracy: 0.9177 Time: 7.59567  | Val Loss: 0.5369 Accuracy: 0.8436\n",
      "Epoch: 121/250 Train Loss: 0.2700 Accuracy: 0.9131 Time: 7.64096  | Val Loss: 0.5802 Accuracy: 0.8352\n",
      "Epoch: 122/250 Train Loss: 0.2663 Accuracy: 0.9150 Time: 7.68971  | Val Loss: 0.5843 Accuracy: 0.8438\n",
      "Epoch: 123/250 Train Loss: 0.2602 Accuracy: 0.9177 Time: 7.61190  | Val Loss: 0.5505 Accuracy: 0.8492\n",
      "Epoch: 124/250 Train Loss: 0.2635 Accuracy: 0.9156 Time: 7.63821  | Val Loss: 0.6170 Accuracy: 0.8311\n",
      "Epoch: 125/250 Train Loss: 0.2605 Accuracy: 0.9137 Time: 7.63250  | Val Loss: 0.5402 Accuracy: 0.8423\n",
      "Epoch: 126/250 Train Loss: 0.2553 Accuracy: 0.9160 Time: 7.67471  | Val Loss: 0.5591 Accuracy: 0.8385\n",
      "Epoch: 127/250 Train Loss: 0.2426 Accuracy: 0.9267 Time: 7.64095  | Val Loss: 0.5437 Accuracy: 0.8443\n",
      "Epoch: 128/250 Train Loss: 0.2456 Accuracy: 0.9232 Time: 7.63526  | Val Loss: 0.5943 Accuracy: 0.8326\n",
      "Epoch: 129/250 Train Loss: 0.2316 Accuracy: 0.9222 Time: 7.61902  | Val Loss: 0.5927 Accuracy: 0.8334\n",
      "Epoch: 130/250 Train Loss: 0.2457 Accuracy: 0.9214 Time: 7.62851  | Val Loss: 0.5534 Accuracy: 0.8499\n",
      "Epoch: 131/250 Train Loss: 0.2492 Accuracy: 0.9227 Time: 7.61857  | Val Loss: 0.5416 Accuracy: 0.8446\n",
      "Epoch: 132/250 Train Loss: 0.2393 Accuracy: 0.9217 Time: 7.57146  | Val Loss: 0.6001 Accuracy: 0.8372\n",
      "Epoch: 133/250 Train Loss: 0.2422 Accuracy: 0.9230 Time: 7.62722  | Val Loss: 0.5707 Accuracy: 0.8395\n",
      "Epoch: 134/250 Train Loss: 0.2292 Accuracy: 0.9262 Time: 7.61778  | Val Loss: 0.5752 Accuracy: 0.8392\n",
      "Epoch: 135/250 Train Loss: 0.2235 Accuracy: 0.9274 Time: 7.61313  | Val Loss: 0.5415 Accuracy: 0.8476\n",
      "Epoch: 136/250 Train Loss: 0.2199 Accuracy: 0.9321 Time: 7.64700  | Val Loss: 0.5958 Accuracy: 0.8359\n",
      "Epoch: 137/250 Train Loss: 0.2224 Accuracy: 0.9268 Time: 7.63738  | Val Loss: 0.5500 Accuracy: 0.8502\n",
      "Epoch: 138/250 Train Loss: 0.2237 Accuracy: 0.9287 Time: 7.66530  | Val Loss: 0.5429 Accuracy: 0.8515\n",
      "Epoch: 139/250 Train Loss: 0.2129 Accuracy: 0.9288 Time: 7.64213  | Val Loss: 0.5555 Accuracy: 0.8431\n",
      "Epoch: 140/250 Train Loss: 0.2180 Accuracy: 0.9296 Time: 7.60435  | Val Loss: 0.6213 Accuracy: 0.8316\n",
      "Epoch: 141/250 Train Loss: 0.2159 Accuracy: 0.9299 Time: 7.64073  | Val Loss: 0.5920 Accuracy: 0.8403\n",
      "Epoch: 142/250 Train Loss: 0.2072 Accuracy: 0.9347 Time: 7.65626  | Val Loss: 0.5936 Accuracy: 0.8367\n",
      "Epoch: 143/250 Train Loss: 0.2071 Accuracy: 0.9312 Time: 7.68148  | Val Loss: 0.5792 Accuracy: 0.8390\n",
      "Epoch: 144/250 Train Loss: 0.1970 Accuracy: 0.9366 Time: 7.64328  | Val Loss: 0.5649 Accuracy: 0.8428\n",
      "Epoch: 145/250 Train Loss: 0.2016 Accuracy: 0.9357 Time: 7.61253  | Val Loss: 0.5787 Accuracy: 0.8469\n",
      "Epoch: 146/250 Train Loss: 0.1963 Accuracy: 0.9390 Time: 7.68622  | Val Loss: 0.5418 Accuracy: 0.8489\n",
      "Epoch: 147/250 Train Loss: 0.2106 Accuracy: 0.9338 Time: 7.64291  | Val Loss: 0.5545 Accuracy: 0.8494\n",
      "Epoch: 148/250 Train Loss: 0.2050 Accuracy: 0.9347 Time: 7.63250  | Val Loss: 0.5725 Accuracy: 0.8494\n",
      "Epoch: 149/250 Train Loss: 0.2081 Accuracy: 0.9336 Time: 7.68126  | Val Loss: 0.6313 Accuracy: 0.8346\n",
      "Epoch: 150/250 Train Loss: 0.2033 Accuracy: 0.9352 Time: 7.61965  | Val Loss: 0.6208 Accuracy: 0.8301\n",
      "Epoch: 151/250 Train Loss: 0.1906 Accuracy: 0.9402 Time: 7.63082  | Val Loss: 0.5810 Accuracy: 0.8415\n",
      "Epoch: 152/250 Train Loss: 0.1844 Accuracy: 0.9410 Time: 7.61295  | Val Loss: 0.6087 Accuracy: 0.8392\n",
      "Epoch: 153/250 Train Loss: 0.1815 Accuracy: 0.9406 Time: 7.59748  | Val Loss: 0.5698 Accuracy: 0.8487\n",
      "Epoch: 154/250 Train Loss: 0.1943 Accuracy: 0.9370 Time: 7.67531  | Val Loss: 0.6121 Accuracy: 0.8395\n",
      "Epoch: 155/250 Train Loss: 0.1815 Accuracy: 0.9408 Time: 7.57123  | Val Loss: 0.5796 Accuracy: 0.8456\n",
      "Epoch: 156/250 Train Loss: 0.1897 Accuracy: 0.9389 Time: 7.61095  | Val Loss: 0.5496 Accuracy: 0.8522\n",
      "Epoch: 157/250 Train Loss: 0.1902 Accuracy: 0.9406 Time: 7.62755  | Val Loss: 0.5770 Accuracy: 0.8484\n",
      "Epoch: 158/250 Train Loss: 0.1811 Accuracy: 0.9429 Time: 7.61989  | Val Loss: 0.5610 Accuracy: 0.8469\n",
      "Epoch: 159/250 Train Loss: 0.1843 Accuracy: 0.9403 Time: 7.64697  | Val Loss: 0.5693 Accuracy: 0.8484\n",
      "Epoch: 160/250 Train Loss: 0.1833 Accuracy: 0.9439 Time: 7.66094  | Val Loss: 0.5656 Accuracy: 0.8454\n",
      "Epoch: 161/250 Train Loss: 0.1706 Accuracy: 0.9464 Time: 7.61171  | Val Loss: 0.5341 Accuracy: 0.8543\n",
      "Epoch: 162/250 Train Loss: 0.1784 Accuracy: 0.9425 Time: 7.66226  | Val Loss: 0.5612 Accuracy: 0.8479\n",
      "Epoch: 163/250 Train Loss: 0.1843 Accuracy: 0.9428 Time: 7.61780  | Val Loss: 0.5462 Accuracy: 0.8561\n",
      "Epoch: 164/250 Train Loss: 0.1753 Accuracy: 0.9452 Time: 7.61671  | Val Loss: 0.5979 Accuracy: 0.8466\n",
      "Epoch: 165/250 Train Loss: 0.1646 Accuracy: 0.9474 Time: 7.61286  | Val Loss: 0.5715 Accuracy: 0.8456\n",
      "Epoch: 166/250 Train Loss: 0.1648 Accuracy: 0.9481 Time: 7.63046  | Val Loss: 0.5611 Accuracy: 0.8497\n",
      "Epoch: 167/250 Train Loss: 0.1748 Accuracy: 0.9476 Time: 7.56253  | Val Loss: 0.5742 Accuracy: 0.8504\n",
      "Epoch: 168/250 Train Loss: 0.1660 Accuracy: 0.9452 Time: 7.55686  | Val Loss: 0.5501 Accuracy: 0.8476\n",
      "Epoch: 169/250 Train Loss: 0.1565 Accuracy: 0.9510 Time: 7.59256  | Val Loss: 0.5465 Accuracy: 0.8540\n",
      "Epoch: 170/250 Train Loss: 0.1536 Accuracy: 0.9511 Time: 7.57952  | Val Loss: 0.5911 Accuracy: 0.8448\n",
      "Epoch: 171/250 Train Loss: 0.1677 Accuracy: 0.9455 Time: 7.61471  | Val Loss: 0.5473 Accuracy: 0.8520\n",
      "Epoch: 172/250 Train Loss: 0.1545 Accuracy: 0.9514 Time: 7.60294  | Val Loss: 0.5760 Accuracy: 0.8489\n",
      "Epoch: 173/250 Train Loss: 0.1536 Accuracy: 0.9505 Time: 7.63757  | Val Loss: 0.5562 Accuracy: 0.8520\n",
      "Epoch: 174/250 Train Loss: 0.1611 Accuracy: 0.9477 Time: 7.57544  | Val Loss: 0.5528 Accuracy: 0.8487\n",
      "Epoch: 175/250 Train Loss: 0.1667 Accuracy: 0.9487 Time: 7.64190  | Val Loss: 0.5676 Accuracy: 0.8476\n",
      "Epoch: 176/250 Train Loss: 0.1670 Accuracy: 0.9481 Time: 7.59834  | Val Loss: 0.5646 Accuracy: 0.8489\n",
      "Epoch: 177/250 Train Loss: 0.1536 Accuracy: 0.9507 Time: 7.62416  | Val Loss: 0.5672 Accuracy: 0.8494\n",
      "Epoch: 178/250 Train Loss: 0.1659 Accuracy: 0.9490 Time: 7.62900  | Val Loss: 0.5608 Accuracy: 0.8476\n",
      "Epoch: 179/250 Train Loss: 0.1474 Accuracy: 0.9530 Time: 7.61791  | Val Loss: 0.5632 Accuracy: 0.8540\n",
      "Epoch: 180/250 Train Loss: 0.1587 Accuracy: 0.9492 Time: 7.66000  | Val Loss: 0.5495 Accuracy: 0.8489\n",
      "Epoch: 181/250 Train Loss: 0.1590 Accuracy: 0.9504 Time: 7.58378  | Val Loss: 0.5595 Accuracy: 0.8494\n",
      "Epoch: 182/250 Train Loss: 0.1563 Accuracy: 0.9510 Time: 7.59682  | Val Loss: 0.5570 Accuracy: 0.8492\n",
      "Epoch: 183/250 Train Loss: 0.1541 Accuracy: 0.9502 Time: 7.67709  | Val Loss: 0.5370 Accuracy: 0.8558\n",
      "Epoch: 184/250 Train Loss: 0.1449 Accuracy: 0.9550 Time: 7.64220  | Val Loss: 0.5514 Accuracy: 0.8530\n",
      "Epoch: 185/250 Train Loss: 0.1532 Accuracy: 0.9489 Time: 7.72061  | Val Loss: 0.5533 Accuracy: 0.8499\n",
      "Epoch: 186/250 Train Loss: 0.1379 Accuracy: 0.9578 Time: 7.60935  | Val Loss: 0.5483 Accuracy: 0.8522\n",
      "Epoch: 187/250 Train Loss: 0.1479 Accuracy: 0.9541 Time: 7.62996  | Val Loss: 0.5438 Accuracy: 0.8545\n",
      "Epoch: 188/250 Train Loss: 0.1465 Accuracy: 0.9518 Time: 7.65995  | Val Loss: 0.5676 Accuracy: 0.8476\n",
      "Epoch: 189/250 Train Loss: 0.1390 Accuracy: 0.9578 Time: 7.64478  | Val Loss: 0.5632 Accuracy: 0.8499\n",
      "Epoch: 190/250 Train Loss: 0.1500 Accuracy: 0.9527 Time: 7.66798  | Val Loss: 0.5634 Accuracy: 0.8507\n",
      "Epoch: 191/250 Train Loss: 0.1380 Accuracy: 0.9546 Time: 7.70819  | Val Loss: 0.5656 Accuracy: 0.8510\n",
      "Epoch: 192/250 Train Loss: 0.1377 Accuracy: 0.9564 Time: 7.64624  | Val Loss: 0.5623 Accuracy: 0.8517\n",
      "Epoch: 193/250 Train Loss: 0.1452 Accuracy: 0.9546 Time: 7.64670  | Val Loss: 0.5583 Accuracy: 0.8499\n",
      "Epoch: 194/250 Train Loss: 0.1367 Accuracy: 0.9562 Time: 7.60055  | Val Loss: 0.5516 Accuracy: 0.8545\n",
      "Epoch: 195/250 Train Loss: 0.1373 Accuracy: 0.9590 Time: 7.59254  | Val Loss: 0.5437 Accuracy: 0.8515\n",
      "Epoch: 196/250 Train Loss: 0.1325 Accuracy: 0.9572 Time: 7.60786  | Val Loss: 0.5677 Accuracy: 0.8497\n",
      "Epoch: 197/250 Train Loss: 0.1360 Accuracy: 0.9579 Time: 7.60991  | Val Loss: 0.5708 Accuracy: 0.8507\n",
      "Epoch: 198/250 Train Loss: 0.1321 Accuracy: 0.9571 Time: 7.59951  | Val Loss: 0.5538 Accuracy: 0.8530\n",
      "Epoch: 199/250 Train Loss: 0.1288 Accuracy: 0.9591 Time: 7.65471  | Val Loss: 0.5618 Accuracy: 0.8561\n",
      "Epoch: 200/250 Train Loss: 0.1229 Accuracy: 0.9616 Time: 7.61064  | Val Loss: 0.5498 Accuracy: 0.8589\n",
      "Epoch: 201/250 Train Loss: 0.1421 Accuracy: 0.9568 Time: 7.69038  | Val Loss: 0.5479 Accuracy: 0.8538\n",
      "Epoch: 202/250 Train Loss: 0.1352 Accuracy: 0.9587 Time: 7.66823  | Val Loss: 0.5489 Accuracy: 0.8555\n",
      "Epoch: 203/250 Train Loss: 0.1280 Accuracy: 0.9605 Time: 7.65378  | Val Loss: 0.5632 Accuracy: 0.8517\n",
      "Epoch: 204/250 Train Loss: 0.1401 Accuracy: 0.9533 Time: 7.66719  | Val Loss: 0.5697 Accuracy: 0.8497\n",
      "Epoch: 205/250 Train Loss: 0.1249 Accuracy: 0.9628 Time: 7.66359  | Val Loss: 0.5544 Accuracy: 0.8494\n",
      "Epoch: 206/250 Train Loss: 0.1382 Accuracy: 0.9575 Time: 7.59201  | Val Loss: 0.5582 Accuracy: 0.8520\n",
      "Epoch: 207/250 Train Loss: 0.1281 Accuracy: 0.9598 Time: 7.68287  | Val Loss: 0.5528 Accuracy: 0.8474\n",
      "Epoch: 208/250 Train Loss: 0.1290 Accuracy: 0.9605 Time: 7.57835  | Val Loss: 0.5482 Accuracy: 0.8522\n",
      "Epoch: 209/250 Train Loss: 0.1237 Accuracy: 0.9609 Time: 7.58122  | Val Loss: 0.5504 Accuracy: 0.8548\n",
      "Epoch: 210/250 Train Loss: 0.1315 Accuracy: 0.9598 Time: 7.61872  | Val Loss: 0.5454 Accuracy: 0.8555\n",
      "Epoch: 211/250 Train Loss: 0.1263 Accuracy: 0.9618 Time: 7.61861  | Val Loss: 0.5456 Accuracy: 0.8561\n",
      "Epoch: 212/250 Train Loss: 0.1280 Accuracy: 0.9624 Time: 7.56277  | Val Loss: 0.5480 Accuracy: 0.8548\n",
      "Epoch: 213/250 Train Loss: 0.1276 Accuracy: 0.9593 Time: 7.58243  | Val Loss: 0.5522 Accuracy: 0.8548\n",
      "Epoch: 214/250 Train Loss: 0.1233 Accuracy: 0.9623 Time: 7.65370  | Val Loss: 0.5494 Accuracy: 0.8512\n",
      "Epoch: 215/250 Train Loss: 0.1215 Accuracy: 0.9611 Time: 7.58430  | Val Loss: 0.5434 Accuracy: 0.8548\n",
      "Epoch: 216/250 Train Loss: 0.1219 Accuracy: 0.9625 Time: 7.59126  | Val Loss: 0.5568 Accuracy: 0.8522\n",
      "Epoch: 217/250 Train Loss: 0.1261 Accuracy: 0.9597 Time: 7.62177  | Val Loss: 0.5587 Accuracy: 0.8507\n",
      "Epoch: 218/250 Train Loss: 0.1282 Accuracy: 0.9597 Time: 7.62678  | Val Loss: 0.5511 Accuracy: 0.8553\n",
      "Epoch: 219/250 Train Loss: 0.1188 Accuracy: 0.9612 Time: 7.64340  | Val Loss: 0.5538 Accuracy: 0.8538\n",
      "Epoch: 220/250 Train Loss: 0.1272 Accuracy: 0.9611 Time: 7.58969  | Val Loss: 0.5556 Accuracy: 0.8535\n",
      "Epoch: 221/250 Train Loss: 0.1174 Accuracy: 0.9653 Time: 7.62534  | Val Loss: 0.5487 Accuracy: 0.8532\n",
      "Epoch: 222/250 Train Loss: 0.1208 Accuracy: 0.9606 Time: 7.62935  | Val Loss: 0.5458 Accuracy: 0.8535\n",
      "Epoch: 223/250 Train Loss: 0.1146 Accuracy: 0.9660 Time: 7.65837  | Val Loss: 0.5444 Accuracy: 0.8563\n",
      "Epoch: 224/250 Train Loss: 0.1161 Accuracy: 0.9642 Time: 7.69822  | Val Loss: 0.5512 Accuracy: 0.8563\n",
      "Epoch: 225/250 Train Loss: 0.1190 Accuracy: 0.9622 Time: 7.66604  | Val Loss: 0.5410 Accuracy: 0.8576\n",
      "Epoch: 226/250 Train Loss: 0.1263 Accuracy: 0.9602 Time: 7.60811  | Val Loss: 0.5384 Accuracy: 0.8558\n",
      "Epoch: 227/250 Train Loss: 0.1147 Accuracy: 0.9639 Time: 7.62599  | Val Loss: 0.5457 Accuracy: 0.8545\n",
      "Epoch: 228/250 Train Loss: 0.1185 Accuracy: 0.9642 Time: 7.70424  | Val Loss: 0.5405 Accuracy: 0.8545\n",
      "Epoch: 229/250 Train Loss: 0.1224 Accuracy: 0.9610 Time: 7.60565  | Val Loss: 0.5481 Accuracy: 0.8540\n",
      "Epoch: 230/250 Train Loss: 0.1205 Accuracy: 0.9631 Time: 7.64658  | Val Loss: 0.5492 Accuracy: 0.8540\n",
      "Epoch: 231/250 Train Loss: 0.1138 Accuracy: 0.9664 Time: 7.71023  | Val Loss: 0.5446 Accuracy: 0.8553\n",
      "Epoch: 232/250 Train Loss: 0.1140 Accuracy: 0.9648 Time: 7.59329  | Val Loss: 0.5481 Accuracy: 0.8545\n",
      "Epoch: 233/250 Train Loss: 0.1210 Accuracy: 0.9645 Time: 7.60587  | Val Loss: 0.5514 Accuracy: 0.8553\n",
      "Epoch: 234/250 Train Loss: 0.1144 Accuracy: 0.9654 Time: 7.60332  | Val Loss: 0.5524 Accuracy: 0.8548\n",
      "Epoch: 235/250 Train Loss: 0.1216 Accuracy: 0.9628 Time: 7.62428  | Val Loss: 0.5491 Accuracy: 0.8578\n",
      "Epoch: 236/250 Train Loss: 0.1142 Accuracy: 0.9643 Time: 7.62442  | Val Loss: 0.5497 Accuracy: 0.8538\n",
      "Epoch: 237/250 Train Loss: 0.1202 Accuracy: 0.9651 Time: 7.61833  | Val Loss: 0.5459 Accuracy: 0.8553\n",
      "Epoch: 238/250 Train Loss: 0.1122 Accuracy: 0.9653 Time: 7.65877  | Val Loss: 0.5479 Accuracy: 0.8532\n",
      "Epoch: 239/250 Train Loss: 0.1208 Accuracy: 0.9615 Time: 7.67127  | Val Loss: 0.5469 Accuracy: 0.8550\n",
      "Epoch: 240/250 Train Loss: 0.1183 Accuracy: 0.9636 Time: 7.61686  | Val Loss: 0.5435 Accuracy: 0.8563\n",
      "Epoch: 241/250 Train Loss: 0.1160 Accuracy: 0.9644 Time: 7.58516  | Val Loss: 0.5484 Accuracy: 0.8566\n",
      "Epoch: 242/250 Train Loss: 0.1183 Accuracy: 0.9648 Time: 7.74081  | Val Loss: 0.5525 Accuracy: 0.8545\n",
      "Epoch: 243/250 Train Loss: 0.1169 Accuracy: 0.9647 Time: 7.64284  | Val Loss: 0.5522 Accuracy: 0.8532\n",
      "Epoch: 244/250 Train Loss: 0.1270 Accuracy: 0.9616 Time: 7.61692  | Val Loss: 0.5485 Accuracy: 0.8563\n",
      "Epoch: 245/250 Train Loss: 0.1148 Accuracy: 0.9632 Time: 7.65781  | Val Loss: 0.5518 Accuracy: 0.8561\n",
      "Epoch: 246/250 Train Loss: 0.1135 Accuracy: 0.9638 Time: 7.68832  | Val Loss: 0.5514 Accuracy: 0.8530\n",
      "Epoch: 247/250 Train Loss: 0.1122 Accuracy: 0.9632 Time: 7.65730  | Val Loss: 0.5537 Accuracy: 0.8525\n",
      "Epoch: 248/250 Train Loss: 0.1139 Accuracy: 0.9662 Time: 7.60198  | Val Loss: 0.5520 Accuracy: 0.8561\n",
      "Epoch: 249/250 Train Loss: 0.1251 Accuracy: 0.9616 Time: 7.59978  | Val Loss: 0.5492 Accuracy: 0.8576\n",
      "Epoch: 250/250 Train Loss: 0.1104 Accuracy: 0.9669 Time: 7.62751  | Val Loss: 0.5555 Accuracy: 0.8530\n",
      "#Parameter: 733514 Accuracy: 0.8529936305732484\n"
     ]
    }
   ],
   "source": [
    "train_dataloader, val_dataloader = build_dataloader(128, train_dataset, val_dataset)\n",
    "net, norm_stats = train_net(\n",
    "    train_dataloader,\n",
    "    val_dataloader,\n",
    "    add_norm=True,\n",
    "    dropout=0,\n",
    "    lr=0.01,\n",
    "    weight_decay=0,\n",
    "    max_epochs=250,\n",
    ")\n",
    "\n",
    "eval_result = eval_image_classifier(net, val_dataloader.dataset, DEVICE)\n",
    "ss = [result.gt_label == result.predicted_label for result in eval_result]\n",
    "print(f\"#Parameter: {count_trainable_parameter(net)} Accuracy: {sum(ss) / len(ss)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7082eac",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pytorch-cu124",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
